Gait is the special pattern of human locomotion. It is fairly unique to an individual due to one's specific muscular-skeletal bio-mechanism. Humans can often recognize acquaintances by the way they walk or jog. However, as a behavioral biometrics, gait may also be affected by transient factors such as tiredness, sickness, emotions, and the like. In addition, external factors such as clothes, shoes, carried loads, and floor characteristics can also influence gait. Such motion analysis would be useful for health monitoring, disease diagnosis, and the like.
Automatic gait biometrics, which studies gait using sensory data, has been an active research area receiving increasing attention over the years. Similar to fingerprints and iris biometrics, gait biometrics can be performed for two purposes: (1) identification, where a gait is compared to a database of enrolled gaits with known identities to determine whom the unknown gait belongs to, and (2) authentication, where a gait is compared to the enrolled gait data of a known person to validate the identity.
Computer vision based gait recognition extracts motion features from image sequences for gait classification. These approaches are, in general, susceptible to variations in viewing geometry, background clutter, varying appearances, uncontrolled lighting conditions, and low image resolutions. Measurements from floor pressure sensors have also been explored for gait recognition. However, these systems are usually too cumbersome to deploy for practical applications.
In the past decade, accelerometers have been intensely researched for gait and activity analysis. These sensors directly measure locomotion when worn on a human body. Such sensors are advantageous compared to both videos and floor sensors for automatic gait biometrics. Vision based approaches must infer body motion from cluttered images. It is highly ambiguous, error prone, and vulnerable to variations in a number of external factors. In contrast, accelerometers directly measure human body motion to achieve more accurate gait biometrics. Accelerometers are also inexpensive, small in size, and very easy to deploy. Mobile devices such as smart phones and tablets use accelerometers to automatically determine the screen layout for improved user experience. In one embodiment of the disclosure, the ubiquity of mobile devices embedded with inertial sensors is used to collect motion data continuously for unobtrusive gait-based authentication and identification.
Accelerometer based gait and activity analysis has been a popular research area since the pioneering work done by Mantyjarvi et al. about a decade ago. As is disclosed in J. Mantyjarvi, J. Himberg, and T. Seppanen, Recognizing Human Motion with Multiple Acceleration Sensors, IEEE Int'l Conf. Systems, Man, and Cybernetics, 2001 and J. Mantyjarvi, M. Lindholm, E. Vildjiounaite, S.-M. Makela, and H. Ailisto, Identifying Users of Portable Devices From Gait Pattern with Accelerometers, IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 2, pp. 973-976, 2005, the contents of which are incorporated herein. Earlier work used multiple motion sensors attached to human body parts to analyze their movements and bio kinematics. Later, data from a single sensor at a fixed position such as the feet, hips, or waist was also exploited. With the proliferation of smart phones equipped with advanced sensors, there has been a surge of research interest on the use of accelerometers in commercial off the shelf (COT) mobile devices for activity and gait classification. Unlike the dedicated sensors used in earlier research, accelerometer signals in mobile devices are usually irregularly sampled at a relatively low frame rate for power conservation and efficient resource sharing.
Triple axis accelerometers capture accelerations along three orthogonal axes of the sensor. Given a multivariate time series of the acceleration data, feature vectors are usually extracted for signal windows corresponding to each detected gait cycle or for windows of a pre-specified size. These windows are compared and matched based on template matching, using either the correlation method or dynamic time warping. Alternatively, statistical features including mean, standard deviations, or time span between peaks in windows, histograms, entropy, higher order moments, and features in spatial domain are also used. Fast Fourier Transforms (FFT) and wavelet coefficients in the frequency domain are used to compare longer sequences. Classifiers including nearest neighbor classifier, support vector machine (SVM), and Kohonen self-organizing map have been used. In some cases, preprocessing such as weighted moving average is applied to suppress the noise in data.
Despite the surge in research efforts, gait biometrics using accelerometers still faces an immense challenge in dealing with variations typical in practical applications. As a behavioral biometric, gait exhibits far more variability than physiological biometrics such as fingerprint or iris biometrics. A person's gait is influenced by his/her physical or psychological status such as emotion, fatigue, well-being, and the like. In addition, external factors such as clothes, shoes, carried loads, and ground characteristics can influence a person's gait. To make it even more challenging, there are huge variations in existing data collection processes. Currently, accelerometers only measure local motion where they are worn, and motion patterns differ from one part of the body to another due to the articulated nature of body motion. Even when the sensor is placed at a fixed location, the data measurements can still change depending on the orientation of the sensors.
Most existing research has been conducted in controlled laboratory settings to minimize these variations. In some cases the sensors are placed in a specific way so that intuitive meanings can be assigned to the data components and exploited for gain analysis. As such, existing methods are susceptible to errors when used in real-world conditions. Although promising results have been reported in well-controlled studies on gait biometrics using accelerometers, there is still a large performance gap between laboratory research and real-world applications. For practical applications, it may be unrealistic to assume fixed placement of the sensor. Mobile devices are usually carried casually in pockets or hands without constraints in orientation. Since the same external motion results in completely different measurements with changing sensor orientation, it is essential to compute gait biometrics robust to sensor rotation for realistic scenarios. However, research on this aspect is rather scarce.
Mantyjarvi et al. used both principle component analysis (PCA) and independent component analysis (ICA) to discover “interesting directions” to compute gait features for activity analysis. The underlying assumption of identical data distributions for both training and testing data are unlikely to hold for realistic applications and computed gait features based on magnitude measurements. The computation of an univariate magnitude series using raw 3D multivariate series resulted in information loss and ambiguity artifacts.
One approach to this challenge has been augmenting the training set with simulated data at multiple sensor orientations by artificially rotating available training data. However, significant artificial sampling was needed to tessellate the 3D rotational space and creates unbearable computational and storage burden with the additional risk of degraded classifier performance. Orientation invariant features were also extracted using the power spectrum of the time series. However, this methodology suffered shortcomings common to frequency domain methods: loss of temporal locality and precision, and vulnerability to drifting in gait tempo. Others have used a co-built-in gyroscope sensor to calibrate accelerometer data to the up-right posture in order to reduce the influence of noise in sensor orientation. This approach requires calibration prior to every data collection, expects the sensor to not rotate during data collection, only relieves noise in the vertical direction, and makes unrealistic assumptions that all poses are up-right.
The previous studies paint a picture of drastic degradation in gait recognition performance in the more relaxed scenarios. Even with the new invariant features, an accuracy of approximately 50% was reported. On the other hand, performances in the high 90s are often achieved in more controlled scenarios. Although each study used its own dataset and evaluation standards so the numbers are not directly comparable, the constant large gap in performance does highlight the challenge in realistic gait biometrics using orientation dependent motion sensors.
Although state-of-the-art accelerometer based gait recognition techniques work fairly well under constrained conditions, their performance degrades significantly for real world applications where variations in sensor placement, footwear, outfit, and performed activities persist. For a mobile device based gait biometrics system to succeed, it is crucial to address the variations in inertial sensor orientation due to casual handling of mobile devices. It is also crucial to address variation in pace and terrain to accurately use gait analysis in real world applications.